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I. Overview 


Urban district reform has been hampered by the challenge of understanding and supporting 
the tremendous complexity of district change. I mproving this understanding through 
actionable, practice- based research is the purpose of this study. 

We began the study with the hypothesis that achieving districts both align their instructional 
systems (standards, curriculum, instructional strategies, professional development, and 
assessment and data) and their systems to support instruction (accountability and planning, 
human resources, specialized student supports, use of funds, technology and governance) 
around a powerful equity goal and a clear vision of teaching and learning. This basic 
framework is laid out in Defining a Comprehensive Aligned Instructional System (CAIS) by 
Louise Bay Waters (2007). 
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Lessons from the CAIS Benchmarking Study 


The purpose of the CAIS Benchmarking Study was to delve deeply into exactly how districts 
accomplish this alignment. Over the two years of the study, we have found that, indeed, the 
CAIS Study districts did focus on structural alignment, but that they also placed 
considerable emphasis on the development of organizational cultures that enabled them to 
address issues of equity and instructional quality in ways that many districts can't. These 
cultures were critical in sustaining the change process long enough for reforms to move 
from the cabinet to the classroom or to scale from a pilot to the district as a whole. 

As we moved beyond a high-level understanding of the alignment of district structures and 
cultures to a more detailed look at the precise levers of district change, the role of district 
leaders in managing this change became clearer. What emerged was a picture of change as 
more of a dance than a march. Like good dancers, CAIS Study superintendents possessed a 
repertoire of moves, and they used these to respond to their changing and district-specific 
contexts. While these moves had many commonalities, their sequence and form varied. The 
most proficient of the superintendents not only had a range of moves, but were very 
intentional in their use and reflective in their application, thereby enlarging their repertoire 
over time. 


How Districts Advance Equity-Based Systems Reform 




Out of these lessons emerged a more complex theory of equity- based district reform. On 
the basis of this study, we posit that high-performing districts that sustain and improve 
academic achievement for low-performing students and students of poverty and color: 

* Focus on an Equity Vision. They have leaders at the top of the system who keep 
the focus on a core strategy or set of strategies related to improving the 
achievement of under-performing students. 

■ Create Catalysts and Leverage Context. These leaders intentionally leverage 
opportunities and evidence to help build stakeholder understanding, mobilize 
renewed efforts, adjust strategy and clarify their goal. 

* Align Cultures and Systems. They have people at multiple levels who implement 
both strategy and tactics in ways that, over time, serve to align both structures and 
culture with the district's goals for students. 
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- Commit to a Sustained Strategy for Change. While there are many levers for 
change, there are three that seem to be of particular significance: building human 
capital, building the discipline of continuous improvement and defining a high-quality 
instructional core. 

The hallmark of these districts' journeys is that they develop and are sustained over a 
period of years as a result of leaders' ability to manage the politics of reform including 
various destabilizing internal and external forces. 
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Lessons from the CAIS Benchmarking Study 


II. CAIS Benchmark District Selection and Research Methodology 

While urban reform has been hampered by the challenge of the complexity of district 
change, studying district- level change has been equally hampered by the challenge of 
studying complex systems over time. A primary and critical constraint is the lack of 
comparable data across states. A second research challenge involves the methodological 
difficulty of making causal connections between multi-faceted, district-level change and 
student-level achievement, especially in the dynamic, politically-charged contexts in which 
classic "experimental design" approaches are impossible. In addition to these hurdles, 
research is hampered by the lack of clear exemplars: while a variety of researchers have 
found that districts do matter for student achievement, no large urban district with 
significant numbers of students of poverty and color and significant size has yet shown 
across-the-board exemplary levels of sustained student achievement. 

The focus on the school district as an important source and locus of change is relatively 
new. The combination of methodological constraints, lack of comparable data sets and 
dearth of clear exemplars have thus far limited our understanding of exactly how districts 
that are intentionally building systems of achievement have made their progress. The 
default has often been to limit studies to the examination of variables that can be measured 
(i.e., class size, teacher credentials) even though these are well understood to be at best 
proxies for the primary levers for change. The research that has examined district reform 
from a broader systems perspective has often yielded high-level findings which are useful 
but which fail to provide the level of implementation detail, context or description of district 
development over time that is needed by district leaders faced with the practical challenge 
of leading change. 

Selection 

Given these constraints and these needs, the CAIS Benchmarking Study intentionally did 
not undertake to define "the one best path to district reform." Instead we sought to deepen 
the discussion around the "how to" of reform by selecting districts for close observation 
that: 


* Had over 25,000 students with at least 35% of these African American and Latino 
and over 20% students of poverty. 

- Were among the highest performers in that demographic within their state and 
resided in states with strong correlations between state tests and NAEP results. 

* Had been recognized for exemplary district systems in more than one area and an 
intentional focus on systems alignment. 

■ Had a strong equity focus. 

■ Had leaders willing to engage in a new kind of collaborative research project. 


Within these parameters, the CAIS Study selected three districts to examine in detail: 
Montgomery County, Maryland; U-46, Illinois; and Elk Grove, California. Each of these 
districts has had success in raising the achievement of underperforming students and has 
been recognized for the development of systems to support this effort. Bellevue, 
Washington, which does not meet our demographic profile, has received wide recognition 
for closing the advanced placement gap and for developing strong district systems. Their 
accomplishments and openness to partnering in the development of the methodology for 
this study resulted in their inclusion as a pilot CAIS district. 
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Methodology 

The CAIS Benchmarking Study was built around the work of interview teams that met with 
individuals and focus groups at the board, district, site and classroom levels. At the district 
level, this included key members from all segments of the district from operations and 
technology to curriculum, special education and student services. During this time, other 
team members visited at least one elementary, one middle and one high school that 
represented high implementation, and another set that represented low implementation of 
the district's key reform strategies (a total of at least six schools). From this initial multi-day 
district visit and the review of collected documents, the interview team and project 
leadership developed a draft timeline and map of the key events, catalysts and strategies in 
the district's reform journey. They also postulated a list of potential cultural attributes, or 
guiding principles, that they felt represented both the tacit and explicit culture of that 
district. 

With this initial framework in place, researchers returned and met with the district leadership team for a 
structured critique of the analysis. The result informed a more detailed sifting of the data. The final set of 
events, catalysts and strategies, as well as detailed descriptions of each, were arrived at collaboratively. 
Because of the complexity of the data collected and the analysis needed, the necessity of building a 
knowledge management tool to support this work became clear early in the project. The resulting tool, 
which we named CAISWorks, combines a relational database with a visual interface that together allowed 
the data to be tagged, sorted and displayed in a variety of ways. Each event in the database is tied to one 
or more of the 11 CAIS structural components (such as curriculum or governance) and one or more of the 
guiding principles (such as transparency or shared responsibility). Each event also has attached to it 
annotations explaining the identified systemic links as well as related artifacts (board memos, newspaper 
articles, assessment rubrics). To aid in analysis, both data points and annotations are searchable. More 
importantly, a graphical user interface allows researchers to view the data represented visually. These 
visual representations include depiction of the overall reform timeline as well as depictions of the evolution 
of various elements in relation to each other (e.g., use of data vs. professional development). Once the 
data was entered and the preliminary analysis complete, the team returned to the district again to use the 
tool with key district personnel to identify missing elements and unearth linkages not yet found by the 


Facilitated by the power of CAISWorks, the CAIS Benchmarking Study was able to put in place a 
methodology that allowed researchers to collaboratively analyze the multi-dimensional process of district 
reform in partnership with the participating districts. The ability to view multiple layers of data and visually 
search for patterns has made possible the co-construction of knowledge between practitioners and 
researchers as well as across the four district research teams. The tool and the embedded artifacts and 
research summaries provide rich case studies on individual district reform. The Cross- District Analysis 
then takes these individual learnings and looks across the districts for patterns that can be gleaned. 

The Cross-District Analysis 

This cross-district analysis assembles a series of the lessons from the three CAIS 
benchmark districts and the Bellevue pilot that cast new light on how diverse, achieving 
districts with an equity focus have intentionally aligned their systems to focus on delivering 
high quality teaching and learning for traditionally underperforming students. Given the 
limitations of the study, these findings must be seen as descriptive of a hypothesis about 
promising practices rather than a prescriptive recipe for reform. Taken together, though, 
the identified practices point in directions that have important implications for leaders both 
within and outside school systems whose focus is on district reform. Though causal 
connections cannot be drawn between specific elements of the reform efforts in each district 
and specific results, the goal of the study was to explore how these specific elements can 
work together to support change for students. The identification of specific elements is also 
an important step toward identifying those variables that merit the development of a 


team. 
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consistent set of measurement metrics that may one day allow the field to take on the 
analytic challenge of understanding exactly which piece of the puzzle made a difference to 
which outcome. In the meantime, a more holistic portrait of promising practices has the 
potential to help the district leaders whose job it is to assemble the pieces of a reform effort 
that can bring closer to reality the goal of providing life changing options for students of 
color and poverty. 
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III. Levers for Improving Access to Quality of Instruction: 

The "What" of District Reform 

Committing to a Sustained Strategy 

In public education today, leaders are looking for a strategy for improving teaching and 
learning. Thus Harvard, in its Public Education Leadership Program (PELP), talks about 
strategy as a key ingredient in leading change in public education. According to Stacey 
Childress, PELP co-founder, "strategy is the set of actions an organization chooses to pursue 
in order to achieve its objectives." 

But a set of loosely connected actions don't add up to a strategy. Strategy is a set of 
"deliberate" actions that are "puzzle pieces that fit together to create a clear picture of how 
the people, activities and resources of an organization can work effectively to accomplish a 
collective purpose." Strategy is not only a road map, it is also a tool for coherence. Strategy 
is something that evolves over the course of many years. While it takes into account new 
information and innovations, it adopts only what advances the strategic thrust of the 
organization. In contrast, in districts where there is no coherent strategy, you see a "flavor 
of the month" approach to reform. As such, as Childress goes on to say, "strategy is about 
choosing — choosing what to do, and just as importantly, choosing what not to do." (PELP 
Coherence Framework, October 2006) 

Based on our research, achieving districts are very intentional about their choice of a key 
strategy or sequence of strategies. They are equally clear on their theory of action, i.e., 
how this strategy leads to improved teaching and learning and the access to that instruction 
for previously underserved students. They build cultures and communication to support the 
strategy and align their systems around it. In the CAIS Benchmarking Study, we have found 
three major strategic levers for this type of change — three foci for reform: 

* Building human capital 

■ Building the discipline of continuous improvement 

* Defining a high-quality instructional core 

While attending to all, the three benchmarking and one pilot district each sequenced their 
overall strategy differently, beginning with the lever that was to remain the district's central 
priority. In Montgomery County, where commissions had highlighted the lack of 
accountability for student achievement, the initial lever was building the discipline of 
continuous improvement. Elk Grove, with its rapid growth, gave greatest focus to building 
human capital, which was sustained over three superintendents. In both Bellevue and U-46, 
new superintendents entered districts with strong cultures of site- based decision making 
and no central curriculum or assessment. While implementing in very different ways, both 
initially undertook to define a high-quality instructional core. Although each of the four 
districts sustained commitment to one preeminent strategy, all developed intentionality in 
each of the three areas over time. The lessons learned about using these levers to align 
systemic change, detailed below, therefore draw from all the districts. 
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IV. Innovative Approaches in Systems Alignment 


All four CAIS Study districts have intentionally focused on systems building in three key 
areas: capacity building, continuous improvement and the instructional core. Going one step 
further, they have also focused on systems alignment between structures, culture and 
vision as a strategy for moving student achievement and equity. Beyond this overall 
systemic approach, they have also developed innovative tactics that embody and enhance 
coherence. These tactics represent specific "lessons learned" with potential applicability in 
other districts seeking to accelerate systemic reform. The innovations fall into three 
categories: redefining organizational boundaries, reframing traditional relationships and 
leveraging technology. 

Redefining Organizational Boundaries 

Loaned Leaders: U-46 . When Superintendent Connie Neale was confronted by the 
combined fiscal and achievement crises at the beginning of her term, she also faced holes in 
her administrative team. Needing to jump-start reform and wanting time to both grow and 
assess potential leaders, she turned outside to fill her immediate needs. With support from 
the Stupski Foundation, she hired a retired deputy state superintendent to lead her 
instructional reform, supporting her with external literacy experts and an internal project 
team of teacher leaders. An external data expert, also Stupski-funded, was hired as a 
consultant to head up data and assessment with an initial focus on English Language 
Learners and low-performing schools. By bringing in people with known expertise who were 
unconnected to the internal turmoil and closely aligned to her vision, she was able to 
accelerate systems building. The subsequent year, Neale created the position of Chief 
Academic Officer and elevated a principal who had been involved with the project team. The 
team continued rolling out the Curriculum Roadmap and the loaned deputy remained as a 
part-time coach and thought partner for the CAO and team for the next four years. That 
second year, she also brought the data consultant inside as Director of Data and 
Assessment, having now established him with teachers and principals. The year of loaned 
leadership provided Superintendent Neale the space to stabilize the system and begin her 
reforms. Ironically, funding district positions is frequently seen as an inappropriate role for 
educational foundations, a premise this research calls into question. 

Teacher Education Institute (TEI): Elk Grove . Faced with burgeoning enrollment and a 
slow pipeline from nearby teacher-credentialing institutions, Elk Grove sought a partnership 
with San Francisco State University in the early 1990s to create the district's own 11-month 
teacher preparation program. To date, the program has produced over 700 teachers who 
have gone on to work for the district. TEI has allowed Elk Grove to put in place rigorous 
quality screens and tailor preparation to their curriculum and culture. It has also supported 
their efforts to recruit non-traditional candidates, particularly candidates of color, outside of 
the local area, bringing them in as interns. Building on the success of TEI, Elk Grove later 
instituted an administrative credential program with Sacramento State University to allow 
them to train their own administrators. A testament to the success of TEI has been its 
strong support through three superintendents, the initiator, Bob Trigg, and subsequently 
Dave Gordon and Steve Ladd. 

Refraining Traditional Relationships 

Bringing Unions into Management: Montgomery County . Jerry Weast's initial 
description of MCPS as a school system divided into two separate and unequal districts — a 
Red Zone of poverty, color and low achievement and a Green Zone of affluence and high 
achievement — led directly into "Our Call to Action," which was designed to mobilize both 
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the district and the community to embrace all students and ensure that achievement would 
not be predictable by race. This same call to move across boundaries and develop shared 
responsibility for all students went out to the three employee unions. Beginning as 
collaboration and building on the district's earlier move to interest- based bargaining, it 
resulted in a restructured governance process. The three unions were invited to participate 
in the district's capital and budget planning process. To ensure all voices were heard, the 
superintendent also invited administrators to form a fourth union to participate on equal 
footing with the others. Once the initial planning processes were completed, the heads of 
the teachers' and administrators' unions became permanent members of the district's 
leadership team. The resulting decision-making structure has allowed MCPS to cross vertical 
and horizontal boundaries within the organization, speak with a consistent voice and model 
a culture of shared responsibility. 

"Neverstreaming": Elk Grove . By the late 1980s, as the population of Elk Grove both 
mushroomed and changed, record numbers of students were being referred for special 
education. In some cases, this included almost a third of the students at a single school. 
Superintendent Dave Gordon, who was closely connected to California's state educational 
leadership in nearby Sacramento, decided to use his insider's knowledge to broker a new 
approach to special education. The agreement he negotiated through state waivers allowed 
Elk Grove to receive funding for a special education population of 10% without testing for 
program eligibility. The funds could then be used in prevention, particularly at the pre- 
school and early elementary levels, and mainstream classroom support — something Elk 
Grove calls "Neverstreaming." "Neverstreaming" has allowed Elk Grove to proactively serve 
special education students while significantly reducing identification. More importantly, it 
has established the value of early intervention and the norm of collective responsibility for a 
traditionally marginalized group of students. The district carefully tracks school-level 
identification data. When identifications seem disproportionately high, it intervenes to 
examine the core program not just the special education services. 

Leveraging Technology 

The Curriculum Web: Bellevue . Bellevue used the creation of a common curriculum as 
the vehicle to meet its goal of providing all students an education preparing them for college 
success. The Curriculum Web emerged as a pivotal structure for developing and supporting 
the common curriculum across subject areas. Essentially, the Web provides an electronic 
portal to the curriculum. It includes a vast array of resources such as lesson plans, pacing 
guides, curriculum materials and even video clips that scaffold the instruction of challenging 
concepts or supply on-demand professional development. At the same time, the Web 
provides an electronic forum for teachers to discuss, share, create and refine curriculum as 
a professional community. In some subject areas, this is highly mediated. In others, it has 
more of a "wiki" tone. The electronic nature of the curriculum also makes it highly 
transportable and enables review and contribution by external curriculum experts. Finally, 
the Web links to the District Data Analyzer that provides access to assessment results, 
transparent to all teachers and administrators. The Curriculum Web both built on and 
reinforced a district culture that valued collaborative innovation, deep understanding of 
subject matter, and transparency of practice and results. By incorporating teachers in 
content refinement the process, it served to diffuse union and teacher resistance to a highly 
mandated curriculum. In other words, it enabled Superintendent Mike Riley to implement a 
"both/and" approach to district reform — one that was both top-down and bottom-up. 
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Instructional Quality Management System (IOMS1: Montgomery County . 

Montgomery County's IQMS was envisioned as a comprehensive system linking curriculum 
resources, student performance data, professional development management and systems 
accountability measures. In the reverse of Bellevue, MCPS began with the data and 
accountability functions of a component of the IQMS — the Instructional Management 
System (IMS). The IMS was designed to enable educators to monitor student progress, 
access state and MCPS standards, analyze assessment data by standard for instructional 
planning, and perform longitudinal analysis of district, school, grade, class, sub-group and 
individual data for program planning. As it grew in sophistication, it included graphic profiles 
showing student performance against targets, similar schools, leading indicators and other 
accountability measures. These measures were in turn linked to accountability processes 
such as the M-STAT, which uses the military model of after-action reviews to facilitate 
school-based data reflection. A later module incorporating curriculum resources was added 
to the larger Instructional Quality Management System along with one focusing on 
professional development. The staff development module integrated course registration, 
management and payroll functions into the system that now enables concurrent analysis of 
student outcomes along with professional development input. While providing significant 
structural supports for the Montgomery County reform process, the IQMS has been equally 
central to building a culture of data use and accountability for student achievement. 
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V. Conclusion 

High performing classrooms and schools, while still not common, can be found across the 
nation. However, these beacons of hope are extremely fragile — difficult to sustain and 
scale beyond heroic teachers and principals. Scale and sustainability, and the equity 
imperative they entail, are dependent on districts. Unfortunately, there are no urban 
districts that have achieved consistently high levels of student performance across 
demographic groups, grade levels and schools over time. For this reason, a focus on 
systems reform remains imperative. 

Given the criticality of this quest, we cannot wait for a comprehensive model to emerge but 
must draw lessons from districts positively moving in this direction. To this end, the Stupski 
Foundation and the Bill and Melinda Gates Foundation undertook the Comprehensive 
Aligned Instructional System Benchmarking Study, in partnership with Springboard Schools, 
to identify common practices in three achieving urban districts that could illuminate 
promising directions to be built upon by other districts seeking to accelerate reform. 

In order to both identify and make these lesson as accessible as possible, it was necessary 
for the CAIS Study to examine district change in three dimensions: across the system — the 
alignment of both district structures and culture; through the system from vision to 
classroom impact; and over time with the dynamics of change. Each dimension involved an 
examination of both "what" they did and "how" they did it. The complexity of the task and 
the desire to make the findings accessible to the wider reform community led to the 
creation of a knowledge management tool, CAISWorks, and to the need for the identification 
of a pilot district, Bellevue, Washington, to partner with in the development of both the 
process and the tools for the larger research project. 

The research sought, and found, ways that leaders managed both the "how" and the "what" 
of equity-based reform in contexts that were often less than ideal. What we have discovered 
is that there is no prescription for change but rather dimensions and processes of systems 
reform that can be intentionally managed. Understanding these dynamics positions leaders 
to accelerate and sustain the development of districts that provide powerful life options for 
students of poverty and color. 

It is our hope that these findings will add to the growing knowledge needed in transforming 
urban education in the United States. It is the belief of our organizations that we in the field 
of educational research and reform must model the types of "collaborative innovation" and 
"transparency" we seek in the districts we support. Though the one best "answer" may 
continue to elude us, we are confident that through our collective work we will build the 
capacity for finding answers to the intransigent challenges of educational equity. 
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